Computing regularization paths for learning multiple kernels
نویسندگان
چکیده
The problem of learning a sparse conic combination of kernel functions or kernel matrices for classification or regression can be achieved via the regularization by a block 1-norm [1]. In this paper, we present an algorithm that computes the entire regularization path for these problems. The path is obtained by using numerical continuation techniques, and involves a running time complexity that is a constant times the complexity of solving the problem for one value of the regularization parameter. Working in the setting of kernel linear regression and kernel logistic regression, we show empirically that the effect of the block 1-norm regularization differs notably from the (non-block) 1-norm regularization commonly used for variable selection, and that the regularization path is of particular value in the block case.
منابع مشابه
Regularizing multiple kernel learning using response surface methodology
In recent years, several methods have been proposed to combine multiple kernels using a weighted linear sum of kernels. These different kernels may be using information coming from multiple sources or may correspond to using different notions of similarity on the same source. We note that such methods, in addition to the usual ones of the canonical support vector machine formulation, introduce ...
متن کاملL2 Regularization for Learning Kernels
The choice of the kernel is critical to the success of many learning algorithms but it is typically left to the user. Instead, the training data can be used to learn the kernel by selecting it out of a given family, such as that of non-negative linear combinations of p base kernels, constrained by a trace or L1 regularization. This paper studies the problem of learning kernels with the same fam...
متن کاملApproximate Regularization Paths for `2-loss Support Vector Machines
We consider approximate regularization paths for kernel methods and in particular `2-loss Support Vector Machines (SVMs). We provide a simple and efficient framework for maintaining an εapproximate solution (and a corresponding ε-coreset) along the entire regularization path. We prove correctness and also practical efficiency our method. Unlike previous algorithms our algorithm does not need an...
متن کاملExploring Regularization in Learning to Predict Structured Objects
Discriminative learning framework is one of the very successful fields of machine learning. The methods of this paradigm, such as Boosting, and Support Vector Machines have significantly advanced the state-of-the-art for classification by improving the accuracy and by increasing the applicability of machine learning methods. However, traditionally these methods do not exploit dependencies betwe...
متن کاملThesis: Multiple Kernel Learning for Object Categorization
Object Categorization is a challenging problem, especially when the images have clutter background, occlusions or different lighting conditions. In the past, many descriptors have been proposed which aid object categorization even in such adverse conditions. Each descriptor has its own merits and de-merits. Some descriptors are invariant to transformations while the others are more discriminati...
متن کامل